An adaptive read/write optimized algorithm for Ceph heterogeneous systems via performance prediction and multi-attribute decision making

Abstract The advent of the Big Data era has brought considerable challenges to storing and managing massive data. Moreover, distributed storage systems are critical pressure capacity costs. Ceph cloud system only selects data nodes based on node capacity. This selection method results in load imbalance limited scenarios heterogeneous systems. Therefore, we add heterogeneity, network state, as performance weights CRUSH algorithm optimize by improving balancing. We designed a model Software Defined Network (SDN) technology. can avoid tedious configuration significant measurement overhead required obtain status traditional architecture. Then propose adaptive read write optimization algorithms SDN Object Storage Device (OSD) is initially classified Node Heterogeneous Resource Classification Strategy. technology used conditions real-time an OSD prediction built for impact factors. Finally, mathematical proposed multi-attribute decision making conjunction with state its model. Furthermore, this addressed adaptively. Compared original system, TOPSIS_PA improves reading operations 36%; TOPSIS_CW TOPSIS_PACW improve elastic 23 60% 36 85%, 180 468% 188 611%, respectively.